Adaptive Resource Management in Distributed Systems

In this dissertation, we focus on resource management in distributed systems. The essence of resource management is to match the requirements of computing tasks with the available resources. We propose and develop approaches to resource management in three qualitatively different systems: (1) server clusters providing computing-as-a-service, (2) tiered-architecture (of servers) hosting web services, and (3) networks of wireless sensors. These systems differ from each other along multiple dimensions: available resources, system dynamics, workload, etc. Still, a common theme in effective resource management for these systems (as demonstrated in this dissertation) is that we must be cognizant of the system heterogeneity (computing resources as well as workload), and adapt to system dynamics. Our work improves upon the state-of-the-art in the three systems in the following way. For systems providing computing-as-a-service, we design and implement a service model that provides predictability in job finish times and prioritized service to delay sensitive jobs. We also develop a machine learning based workload characterization technique for web services that categorizes users’ request based on their resource usage. Such categorization is useful in improving the accuracy of performance models for these systems. In the context of wireless sensor networks, we make the following two contributions: (1) we design an online algorithm that makes joint compression and transmission decisions to save energy, and (2) we explore techniques for detecting anomalies in data collected using these networks.